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WHAT IS THIS REPORT ABOUT? 


n December 2000, the Southwest Educational Development 


Laboratory (SEDL) was awarded a five-year research and development The purpose of this report is to 

(R&D) contract to develop and refine a systemic model to improve present findings based on analyses 
student achievement in reading or mathematics in low-performing districts of more than 260 interviews with 
and schools. The Charles A. Dana Center at The University of Texas at teachers, administrators, parents, 


and other staff and field notes of 
SEDL coordinators who worked 
with the districts. 


Austin and American Indian Research and Development, Inc., are 
partnering with SEDL on this work. SEDL expects that as districts and 
schools increase their capacity to work systemically, student achievement 
will increase. 


At the onset of the project, 20 districts in Arkansas, Louisiana, New Mexico, 
Oklahoma, and Texas in which one or more of the schools were having 
difficulty meeting state expectations for student achievement agreed to 
participate. At the end of 2002, 16 districts and 29 schools within those 
districts were working with SEDL (early on four districts decided to 
discontinue and were replaced). 


The purpose of this report is to present findings based on analyses of more 
than 260 interviews with teachers, administrators, parents, and other staff 
and field notes of SEDL coordinators who worked with the districts. ? 


1 This publication was produced in whole or in part with funds from the Institute of Education 
Sciences, U.S. Department of Education, under contract number ED-01-CO-0009. The 
content herein does not necessarily reflect the views of the U.S. Department of Education, 
any other agency of the U.S. Government, or any other source. 


2. For the purposes of this report, we do not differentiate between types of participants (teachers, 
administrators, parents, or other staff) in calculating the results. Responses from all participants 
are grouped at the district level; thus, all persons regardless of their role in the school system will 
be referred to as “educators.” The one exception to this is in the section on leadership. 
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WHAT DOES WORKING SYSTEMICALLY MEAN? 


Before we present the findings, let’s quickly review SEDLs Working 
Systemically Model. Working systemically means that the whole district, as a 
system, will work together to improve student achievement in reading or 
mathematics. SEDL is guiding this process. 


Working systemically means that the The model has three distinct features: 

whole district, as a system, will work 

together to improve student 1. Multiple levels of the educational system (i.e., district, school, classroom) 

achievement in reading or must be involved. 

mathematics. SEDL site 

coordinators and specialists help 2. All parts or components of the system must be considered in making 

district and school teams through improvements (i.e., standards, curriculum and instruction, assessment, 

each of the five stages of the model. policy and governance, professional staff, resources, and family and 
community). 


3. Five competencies must be mastered by those individuals involved in the 
improvement effort (i.e., collecting, interpreting, and using data; creating 
coherence; forging alliances; building capacity; and promoting innovation). 


SEDL site coordinators and specialists help district and school teams through 
each of the five stages of the model: 


1. Site entry/Data scan - districts identify a general problem from data related 
to reading or mathematics achievement with the help of a SEDL site 
coordinator. 


2. Systems exploration - districts and schools look more closely at data-driven 
problem areas and develop a root cause problem statement that will help 
determine what needs to be accomplished for improvement in reading or 
mathematics. 


3. Planning action - districts and schools develop an ideal statement or overall 
goal from which to develop a system plan and action elements as part of 
the plan. 


4. Taking action and monitoring results - districts and schools implement the 
g g p 
plan and its action elements, monitoring and refining the plan on a regular 
basis. 


5. Recycling the process - districts and schools reassess needs and success with 
the plan, develop a new plan, implement and monitor that new plan on an 
ongoing basis. 
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STUDY QUESTIONS 


The following questions are addressed in this report: 


1. At what stage of the model districts at the end of December 2002 after one 
to two years of work with SEDLs Working Systemically Model? 


2. What factors have emerged as key issues in continuing to work with 
districts and schools? 


FINDINGS 


Progress Through the Model 


SEDL recruited 17 districts in 2001 and another 3 in 2002. Four districts 
chose to discontinue, leaving 16 districts working with SEDL through 2002. 
Each district nominated one or two schools to work with the project. District 
leaders selected the schools with some input from SEDL staff, who encouraged 
district leaders to consider student achievement and feeder patterns. Of the 16 


SEDL recruited 17 districts in 2001 
and another 3 in 2002. Four districts 
chose to discontinue, leaving 16 
districts working with SEDL through 
2002. The progress of districts and 


ee schools in relation to the model was 
are high schools. Three districts have only one participating school. In four determined by analyses of SEDL site 


small districts, all schools are involved in the project. coordinator field notes and interviews. 


districts, 13 have two schools participating in the partnership, two with pairs at 
the same level (i.e., elementary) rather than across levels. Of the 29 
participating schools, 12 are elementary schools, 6 are middle schools, and 11 


The progress of districts and schools in relation to the model was determined 
by analyses of SEDL site coordinator field notes and interviews. Based on the 
information presented in Table 1, it is clear that districts and schools progressed 
through the model’s stages at different rates. 


Table 1: Sites at each Stage of the Model by Start Date 
(Fiscal Year and Quarter) as of December 2002 


FY01 FYO1 FYO1 FY02 FY02 
2nd Qtr. | 3rd Qtr. | 4th Qtr. | 2nd Qtr. | 3rd Qtr. Totals 


Panning Action 


ee 


fRecycling = 
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Based on SEDL site coordinator 
field notes and interviews, a number 
of factors influenced progress 
through the model stages, such as 
the frequency of district team 
meetings, steady attendance at those 
meetings, or consensus on a focus 
from which an action plan could be 
developed. 


Based on SEDL site coordinator field notes and interviews, a number of factors 
influenced progress through the model stages, such as the frequency of district 
team meetings, steady attendance at those meetings, or consensus on a focus 
from which an action plan could be developed. Given the variety of districts 
and schools that SEDL is working with, varability in the rate of progress 
through the model stages among the sites is expected. 


However, districts that developed an early focus on either reading or 
mathematics created action plans more quickly. Six of the 16 districts reported 
a focus on reading or mathematics early in their discussions and moved ahead 
to formulate plans (one district had a plan focused on another area). 
Developing an early focus on an instructional goal contributed to taking action 
more quickly. 


In the nine districts where there was no plan, only one had a focus on reading 
or math. Some districts or schools had started taking action while plans were 
still in discussion, or were taking action in one school but not at the district 
level or across schools. While this may have addressed an immediate need 
expressed by the school or district team in a piecemeal form, it had little impact 
on the system. In some districts and schools, plans focused on issues like 
parental involvement, discipline, or attendance rather than instructional goals. 


Focus Areas 


As part of the process of developing plans, the districts are working on 
techniques or skills to help them accomplish the goals of the Working 
Systemically Model. The following is a list of the 5 skills or competencies and 
the number of districts focusing on each: 


¢ Building capacity - 16 districts are strengthening their capacity to improve 
student achievement 


¢ Using data - 14 districts are developing their ability to work with data 


¢ Forging alliances - 13 districts are reaching out to work with other parts of 
the system 


* Creating Coherence - 11 districts are working on aligning curriculum, 
instruction, and assessment to standards 


¢ Promoting innovation - | district is investigating new programs. 


The districts will use the skills discussed above to develop and implement plans 
to strengthen parts of the school system. The following is a list of focus areas 
and the number of districts working in each area: 


* Curriculum and instruction - 12 districts are focusing on improving 
curriculum and instruction 


* Assessment - 11 districts are working on assessment 


* Professional development - 8 districts are utilizing professional development 
as an important piece of their plan 


¢ Standards - 6 districts are focusing on working with standards 


Progress of Leadership Teams 


All project work is centered on leadership teams. SEDL is training the districts 
and schools to develop the skills to continue the process after the contract ends. 
Teams should cross levels or be reciprocal. Cross level teams have members of 
the school team on the district team or members of the district team on the 
school team but not both. Reciprocal teams have members of the school team 
on the district team and members of the district team on the school team. As 
schools interpret plans made at the district level, district representation helps to 
clarify goals as they apply to the schools. In the reverse, as districts develop 
activities across the system, school representation allows for better information 
and alignment. Where efforts are not coordinated, improvements have less 
impact and receive less support across the system. 


As of December 2002, 12 districts had district-level teams. Districts that were 
still early in their interactions with SEDL and the Working Systemically Model 
may not be at the stage where a district team has been formed. Chart 1 shows 
the number of districts that have reciprocal teams and the number with cross 
level teams. By the end of 2002, half of the districts had reciprocal 
representation on school and district teams. 


Field notes indicate that working on teams and developing reciprocity across 
teams was difficult for some districts and schools. Further, some teams were not 
able to maintain focus when SEDL was not present—they depended on SEDL 
to provide direction and maintain momentum. 


16 Chart 1: Leadership Team Types by Number of Districts 
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Definitions: 


Reciprocal - on both schools and 
district teams 


Cross Level Type 1 - schools on 
district but not on reverse. 


Cross Level Type 2 - district on 
schools but not on reverse 
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SEDL considered how districts and 
schools were addressing these 
factors: 1) Is the curriculum aligned 
to standards? 2) Are resources 
sufficient for instruction? 3) Did 
educators feel that they could be 
successful with all students or only 
part of the student group? 


SUMMARY 


Progress through the Model 


While the information presented in this report reflects work in progress, a 
number of general conclusions can be drawn from the data (as of the end of 
2002): 


1. Half of the districts and schools are in the taking action stage, i.e., they are 
implementing some part of a plan at district or school levels. 


2. Three-fourths of the districts and schools are building capacity through some 
combination of curriculum, instruction, and assessment activities supported 
by professional development. 


3. SEDL coordinators guide most of the work at districts and schools in 
conjunction with local staff (this is part of the model for training local staff). 


4. Few districts showed strong connections between the work of the district 
teams and that of the school teams. 


5. Many team roles and relationships are still developing; many teams show a 
need for SEDL support to maintain focus. 


WHAT OTHER FACTORS EMERGED AS KEY ISSUES? 


This section describes issues that emerged as SEDL reviewed the progress of 
districts and schools. 


Studies indicate that the following areas are important to the success of 
improvement eftorts—alignment of district curriculum with state standards, 
sufficient resources to support core instruction and student achievement, and 
high expectations held by district and school staff for student achievement (see 
end of report for references). SEDL considered how districts and schools were 
addressing these factors: 1) Is the curriculum aligned to standards? 2) Are 
resources sufficient for instruction? 3) Did educators feel that they could be 
successful with all students or only part of the student group? 


Alignment of District Curriculum with State Standards 


Curriculum alignment to state standards helps promote student achievement. 
Chart 2 shows the progress of curriculum alignment to state standards based on 
educator responses. Educators in 8 districts reported that curriculum alignment 
to standards is completed, in 5 districts curriculum alignment is under way, and 
in 3 districts no efforts are being made to align the curriculum. However, when 
asked if instruction was aligned to the curriculum, responses varied greatly. Some 


recognized that the curriculum was aligned to state standards and it was being 
taught in the classroom; others saw it as aligned but questioned whether it was 
being taught in the classroom. 


Chart 2: Progress of Curriculum Alignment to Standards by Number of Districts 


16 


Number of Districts 


Progress 


COMPLETED UNDERWAY NO EFFORT 


Resources 


SEDL asked educators whether district resources were sufficient to support 
classroom instruction. Regardless of the economic status of the district, most 
answered “yes” to this question. Chart 3 (page 8) shows the level of resources 
available for instruction as either adequate or inadequate by number of 
districts. The first bar in the chart indicates that 14 of the 16 districts rated 
the level of resources for instruction as adequate. The second bar indicates that 
only 2 districts rated the level of resources for instruction as inadequate. 
However, when researchers compared resources across districts, some districts 
clearly have more resources than others. Educators commonly reported that 
the resources in their own districts were sufficient to meet their needs 
regardless of outside comparisons. 
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SEDL asked educators whether 
district resources were sufficient to 
support classroom instruction. 
Regardless of the economic status 
of the district, most answered 
“yes” to this question. 


Progress Report to Districts and Schools 


Expectations can influence how 
districts address student outcomes. 
Most educators initially responded 
that they generally held high 
expectations for students. 


Chart 3: Resources for Instruction by Number of Districts 


Number of Districts 
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Educators described how changing budgets have reduced curriculum materials 
to the bare necessities and, in a number of cases, reduced what monies were 
available per pupil in the past. 


Expectations 


Expectations can influence how districts address student outcomes. For 
instance, if expectations are high for only a few students rather than for all 
students, educators might not press all students to achieve. Most educators 
initially responded that they generally held high expectations for students. 


Chart 4: Number of Districts with High Expectations for Some or 
a Few Students 
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However, when asked whether these expectations were held for all students, 
more variation arose in educator responses. Chart 4 (page 8) indicates that 
educators in 12 of the 16 districts had high expectations for some students. 
Only four of the districts reported having high expectations for only a few 
students. No districts reported having high expectations for all students. 


In some cases, educators said that many of the students did not have sufficient 
help at home to succeed; in other cases, they said that students’ native language 
or culture interfered. A number of educators felt that the students had difficult 
lives, and they did not want to make it even more difficult for them. Many 
educators said that regardless of circumstance, they felt it was their 
responsibility to help students succeed. 


Leadership 


Leadership is not an explicit element of SEDLs model, but it is strongly 
associated with the outcomes of reform efforts. Leadership changes have 
occurred in half of the districts and schools, some more than once, in the 
relatively short period that they have been engaged with SEDL. Two districts 
have had changes at the superintendent level; another two districts have had 
changes at both the superintendent and principal levels; three other districts 
have had changes at the principal level; and another district lost a district staff 
leader. It is difficult to maintain reform efforts without some stability of 


leadership. 


Leadership stability is not the only concern, however. A consistent theme that 
emerged during the first two years of work was the ability of district and school 
leaders to diagnose needs, to plan ahead, and to do necessary follow-up to 
support improvement in the long term. Too often, leaders were overwhelmed 
by the daily problems that confronted their districts and schools. Across the 16 
districts, SEDL looked at the ability of leaders to prioritize instruction over 
operation, to be proactive instead of reactive, and to follow through on 
improvement agendas. 


Instruction plus management. While some superintendents combined 
instructional leadership with operational management, all were stronger 
operational managers than instructional leaders. In contrast, six out of 29 
principals fell into the instructional leadership category. When leadership 


becomes more involved with instruction, instruction is likely to improve. 


Proactive or reactive. Another leadership theme is related to whether leaders 
were proactive or reactive in their work with SEDL. Proactive means that they 
planned how to address issues, were organized, had procedures in place, and 
anticipated changes in the district or schools. Reactive means that they operate 
in more of a crisis management mode and were caught by surprise when 
changes occurred. Two of the 16 superintendents were characterized as 
“proactive,” while seven of the 29 principals were considered proactive. This 
means that the majority of leaders in these districts and schools were 
responding to needs on a day-to-day basis. 
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A consistent theme that emerged 
during the first two years of work was 
the ability of district and school leaders 
to diagnose needs, to plan ahead, and 
to do necessary follow-up to support 
improvement in the long term. Too 
often, leaders were overwhelmed by the 
daily problems that confronted their 
districts and schools. 


Progress Report to Districts and Schools 


If student achievement is going to 
improve, then leadership must 
increase the attention they give to 
instructional concerns, be proactive, 
and follow through to ensure 
objectives are realized. 


Based on SEDL’s progress review, 
site coordinators have made some 
refinements to the model to increase 
its responsiveness to the needs of sites 
and enhance the likelihood that 
educators will successfully use the 
model now and in the future. 
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Follow-through and attentiveness. Another leadership theme was whether 
leaders were able to provide follow-through and attended to the work needed. 
Based on field notes, interview data, and interactions with the SEDL leadership 
team, the principals’ and superintendents’ leadership styles were identified 
according to their demonstrated level of follow-through and attentiveness. Of 
the 16 superintendents, 2 superintendents were considered to be high in 
attentiveness, 2 moderate rate, and 12 low. Of the 29 principals, eight were in 
the high category, six in moderate, and 15 in the low category. 


These data show the challenge for leaders in these districts and schools. If 
student achievement is going to improve, then leadership must increase the 
attention they give to instructional concerns, be proactive, and follow through 
to ensure objectives are realized. What superintendents spend their time on 
sends a clear message to principals. The same is true for the messages principals 
send to classroom teachers. 


WHAT CAN WE LEARN FROM THESE DATA? 


This report describes SEDLs work with 16 districts and 29 schools. This work 
focuses on the development and refinement of a systemic model to support 
improvement in student reading or mathematics achievement. 


Over the one to two years of implementation, SEDL and the 16 districts have 
made substantial progress in implementing the model. All of the districts and 
schools are involved in improvement efforts, although there is great variability 
in how directly or quickly their efforts will impact student achievement. SEDL 
is continuing to support and focus the work more directly on student 
outcomes. 


Based on SEDU’ progress review, site coordinators have made some refinements 
to the model to increase its responsiveness to the needs of sites and enhance the 
likelihood that educators will successfully use the model now and in the future. 
Some of the refinements implemented or under consideration follow: 


¢ Increase the time for discussion between SEDL and a district before making 
a commitment to work together, so that both can learn more about each 
other and understand better what the process will be. 


¢ Begin work at the district level first, with representation from schools, and 
then progress to the schools to create a systemic approach and make sure 
that work is coordinated and aligned across the system. 


¢ Focus on reading or mathematics first rather than other areas that may also 
be of concern. 


* Consider ways to address educators’ leadership skills. 


¢ Examine expectations for students in ongoing work with districts and 
schools. 
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As the SEDL five-year project of work extends into 2005, SEDL coordinators 

are continuing to engage districts in an ongoing process to improve outcomes This report finds that schools and 

for their students. Specifically, the information presented in this report has districts are off to a good start in the 

reinforced the following themes: five-year process of implementing 
the SEDL model and addressing 
student achievement. SEDL 
coordinators have revised and 


¢ Leadership, especially instructional leadership, is critical to the process. adjusted their work based on the 
input and responses of the teachers, 
¢ What gets focused on gets done. Whether it is about instruction or administrators, parents, and 
support, a clear and simple focus makes planning and taking action community members across the 
more effective. districts and schools. 


¢ Change is most likely to succeed when all parts of the system support 
each other, especially around issues of curriculum, instruction, and 
assessment related to student achievement. 


¢ Training and skill building (building capacity) is part of a change 


process. 


This report finds that schools and districts are off to a good start in the five-year 
process of implementing the SEDL model and addressing student achievement. 
SEDL coordinators have revised and adjusted their work based on the input 
and responses of the teachers, administrators, parents, and community 
members across the districts and schools. This report documents this progress 
to December 2002. Subsequent reports will add to our knowledge. 


1] 
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WHAT DOES THIS MEAN FOR MY DISTRICT? 


Although the results presented in this report summarize the progress of work 
with SEDL across the 16 districts, the findings are not representative of every 
district and school. The following questions are intended to help district and 
school staff reflect on the work they have done in their own districts or schools 
in relation to the SEDL model and build their capacity to work systemically. 


1. What does it mean to work systemically? How are district and 
school teams collaborating and working across levels? What 
Although the results presented evidence do you have to support your conclusion? What could be 


in this report summarize the done to improve collaboration among the teams? 
progress of work with SEDL 


across the 16 districts, the 2. To what degree does your district's and school’s use of resources, 

findings are not representative curriculum, instruction, and assessment support student 

Bag Ct and school achievement? How might resources be optimized to improve the 
use of standards, curriculum, instruction, and assessment? 


3. In what ways are data used to make decisions about classroom 
instruction? What sources of assessment data are used? What do 
teachers need to know to use assessment data to guide instruction? 


4, What roles do district and school leaders play in improving reading 
and/or mathematics instruction? What roles do leaders play in 
improving student achievement? What roles do leaders play in 
supporting teachers? 


5. How might you ensure that all district and school personnel have 
high expectations for all students? How are high expectations 
communicated to teachers, staff, and students? 
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For further information, or information on SEDI publications and research, contact 
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Southwest Educational Development Laboratory (SEDL) 
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Phone: (800) 476-6861 
E-mail: jbuttram@sedl.org 


Web site: http://www.sedl.org/rel/ 
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